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Abstract This presentation describes work to integrate a set of tools to support early model- 
based analysis of failures and hazards due to system-software interactions. The tools perform 
and assist analysts in the following tasks: 1 ) extract model parts from text for architecture and 
safety/hazard models; 2) combine the parts with library information to develop the models for 
visualization and analysis; 3) perform graph analysis and simulation to identify and evaluate 
possible paths from hazard sources to vulnerable entities and functions, in nominal and 
anomalous system-software configurations and scenarios; and 4) identify resulting candidate 
scenarios for software integration testing. There has been significant technical progress in 
model extraction from Orion program text sources, architecture model derivation (components 
and connections) and documentation of extraction sources. Models have been derived from 
Internal Interface Requirements Documents (lIRDs) and FMEA documents. Linguistic text 
processing is used to extract model parts and relationships, and the Aerospace Ontology also 
aids automated model development from the extracted information. Visualizations of these 
models assist analysts in requirements overview and in checking consistency and 
completeness. 
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Problem 


• NASA needs early evaluation of software-system 
integration risks and constraints 

- Assess system faults, failures and hazards that may 
challenge software in system integration testing 

- Identify robustness and safety issues early 

- Identify requirements gaps early 

• Process of reviewing various large and 
uncoordinated source documents is difficult 



Operations and Stresses 
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Approach and Relevance 



• Semi-automated modeling for Safety Analysis 
and to identify cases for Integration Testing: 

Documents -> Extract Text Construct Model and 
Visualization Analyze Hazard Paths and Simulate 

- Focus on system integration, interfaces, failures and 
hazards, which cause most of aerospace software 
(requirements) defects 

- Focus on information from Preliminary Design 
Review (PDR) - benefit of early analysis is greatest 

• Two Constellation Crew Exploration Vehicle (CEV) cases 

- Launch Abort System (LAS) pyrotechnics and Crew Module 
(CM) 

- Service Model (SM) propulsion 
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Summary of Products 



• Models constructed from information 
extracted from text documents 

• Visualizations for insight into information 
scattered in large documents 

• Component model templates 

• Output for model reuse 
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Latest Results 



• Model Information Extraction from Text 

- Variety of types of documents analyzed 

- Variety of information types extracted 

• Model Construction 

- Component-Connection Models and 
Visualizations 

- Model templates for path analysis 
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Overview of Method and Tools 


• Develop system connection model and visualization 

- Acquire PDR-level documents 

• Interface requirements, failure modes and effects analyses, 
hazard reports 

- Automatically extract needed model information 

• Document analysis and linguistic analysis 

- Semi-automatically construct model, visualization and 
traceability information 

• Nomenclature ontology and component templates library 

• Export the information for reuse 

• Perform path analysis and simulation to analyze 
potential hazard paths 
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Source Documents and Cases 


f 


Documents 

- Failure Mode and 
Effects 

Analysis/Critical 
Items List 
(FMEA/CIL) 

- Internal Interface 
Requirements 
Document (IIRD) 

- Hazard Reports 

Challenges 
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- Variety of Formats 

- Document Maturity 

- Quality of the data 


FMEA/CIL Worksheet for Thruster Mounting Structure 


Thrusters 


■ Thruster 
Mounting 
Structure 


SM 

itructure 
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Semantic Text Analysis Tool (STAT) Extractions 


System to subcomponent indentured hierarchy 

- From FMEA/CIL document front matter section organization 

- From FMEA/CIL worksheet hierarchy: System/Element, Module, Subsystem, 
Sub-Subsystem 

- From FMEA/CIL worksheet failure modes and cause description: Item 
subcomponents 

- From Hazard Report cause descriptions and cause controls 

Components, connections and connection content 

- From lIRDs: Provide and receive statements 

- From FMEA/CIL worksheet Item function description: Provide, receive, transfer 
statements 

- From sensor names, e.g., “Flange Temperature Sensor” 

Function, failure and phase Information 

- From lIRDs: Item vulnerabilities and limits, operational context 

- From FMEA/CIL worksheet: item functions/actions, failure mode description, 
cause description, mission phase 

- From Hazard Report causes descriptions and cause controls 

Acronyms 

Traceability Information: Source document and source texts 
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Linguistic Extraction Progress 


Approach: Parse words and phrases in document text 

- Specify relevant sections and fields for analysis, using document 
structure grammar 

- After linguistic analysis, use Aerospace Ontology nomenclature to 
identify phrases that indicate problems and verbs that indicate 

• Actions/Functions 

• Connecting relationships - e.g., sends, supplies, transfers, distributes, carries 

• Part-of or other structural relationships - contains, consists of, comprises 

- XML-formatted output of relevant model information 

Progress 

- Extraction from multiple document data structures and mime types 

• General format specification approach 

- Better linguistic analysis for information extraction 

• Integrated advanced parser from University of Central Florida 



Thruster 

Mounting 

Structure 


structure 

ri hH 
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LAS Pyro Visualization from FMEA 


Attitude Control Motor (ACM) Abort Motor (AM) Jettison Motor (JM) 


□ □ □ 



□ _>, □ □ 


AM TBI-1 AM Igniter-1 AM FCDC-2 



□ □ □ 


JM FCDC-2 JM FCDC-1 JM NSI-2 



Safe & Arm Components Manifolds 


ACM Controllers 


□ □ 

JM Safe & Arm AM/ACM Safe & Arm 



□ □ 
3-Port Manifold-2 3-Port Manifold-1 



□ □ 

ACM Controller-2 ACM Controller-1 



Numbered multiple instances of components 

Pop-ups on components and connections, with 
model information and traceability 

SAS 09 Technical Malin Automated Tool 


TBI - Through Bulkhead Initiator 
NSI - NASA Standard Initiator 

FCDC - Flexible Confined Detonating Cord (a network) 
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From LAS Case to CEV SM 



• Generalized to another Orion case: 
Service Module (SM) Propulsion 

- PDR data book had updated FMEA/CILs 
and Hazard Reports for extraction 

- Documents for other subsystems were 
generally less complete and less mature 

• Identified and met new challenges 

- New FMEA/CIL worksheet format, Hazard 
Report format, new text styles 
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^ SM Propulsion Case Results 
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Model Construction 


• Hazard Identification Tool (HIT) automatically 

processes extracted XML 

- Uses component hierarchy to define model hierarchy and 
inner models 

- Generates component-connection models, using ontology to 
identify types of components, connections and flows on 
connections 

- Associates with components and connections: functions, 
hazards, failures and traceability information 

• Visualization for Safety personnel 

• XML output for model information reuse 
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LAS Information Extraction 


• Top-Level Model from IIRD Document 




INNER -4 
MODEL 

CM 


"h INNER 
^ MODEL 

LAS 


INNER 

MODEL 

SM 


Text extraction and screening against the ontology 


"The CM shall receive health and status datajrom the LAS in accordance with TBD-LAS- CM- 003 7" 

(HSBSBB& "LAS " :M AT CH ES (#< C ON C EPT- C LAS S 11 At. ort_Sy stem >)) 

" CM 11 :M AT CH ES (#< CO N C EP T- C LAS S 11 A ero sp ac e_Sy stem" >)) 

GQJ&E&X "DATA" :MATCHES (#< CONCEPT- CLASS " Inf ormation_or_Signal_Obj 11 >)) 

I'MM t “re c e ive” MAT CH ES (#< CO N C EP T- C LAS S " Re c e ive " >))) 
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Pop-up Documentation for Connections 


Box pops up when 
user clicks on 
connection arrow 

Information 

• Document Title 

• Requirement 
number 

• Type of thing sent 
: Info/Signal 

• Source Text 

Multiple interface 
requirements 
describe this 
connection 

• Provide version 

• Receive version 


DOCUMENT TITLE: 

Internal Interface Requirements 
Document I'IRD'i 

I 

Launch Abort System (LAS) to Crew 
Module, 'Service Module (CMSM) IRD 


IF .CM .LAS .0052 

Type of thing sent: lnformation_or_Signal_Obj 

Source text: The LAS shall provide health and status data to the CM in 

accordance with TBD-LASCM- 

0037. 

IF .CM .LAS .0053 

Type of thing sent: lnformation_or_Signal_Obj 

Source text: The CM shall receive health and status data from the LAS in 
accordance with TBD-LASCM- 
0037. 


IF .CM LAS .0056 



Text analyz id on 
previous slide 


A connection from the LAS to the CM 
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Pop-up Documentation for Component 


Box pops up when user 
clicks on component 

Information 

- FMEA Document Title and 
FMEA worksheet number 

- Item Name: ...Sensor... 

- Item Function: Senses ... 
temperature and 
provides output to the 
vehicle interface 

- Failure Modes: Loss of 
Thermal Contact. . . 

- Causes for each failure 
mode 

- Sub-component: Fastener 



Component: temperature sensor for a flange 
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Evaluation by Safety Engineer 


• Evaluation session with K. Chen, responsible for Orion avionics 
safety 

• Positive Reactions to Visualization 

- Helps analyst look for missing information in the documents in an 
organized and efficient way 

- Helps analyst check if hazard path is correct and whether fulfills 
requirements 

- Combining extractions from URDs, FMEAs and detailed Hazard 
Analyses can help build a complete picture and identify 
missing and inconsistent information 

• Identify things appearing in the FMEA but not Hazard Analysis 
and vice versa 

• Looks forward to taking this combined information to his safety 
engineer and the Orion contractor 

- References to source documents are helpful 

- This tool should interest NASA headquarters. 

• K. Chen has provided detailed LAS System Hazard Reports for 
model extraction to get the combined picture 
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From FMEA to Hazard Analysis 



• Extraction from Hazard Report text for the Hazard 
Identification Tool (HIT) models and visualizations 

• Extractions from Orion Hazard Report: “Failure to 
Fire Electrically Controlled Pyrotechnics results in 
Loss of Crew/Loss of Mission” 

- Cause B: Avionics/Electrical Failure 

- Cause B description example (4 causes are listed) 

• “A failure in the Test Port Flight Cap prevents power or 
redirects power through a short circuit causing no power to 
reach the NS I.” 

- Cause B controls example 

• “RIU Test Port Flight Cap is designed to prevent shorts of one 
or more firing lines.” 
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Hazard Report Extraction Examples 


- Extraction from Cause Description and Cause controls 

- Components, sub-components, connections, entity in 
Path 

• “RIU Test Port Flight Cap, NSI, firing lines, power” 

• Others: “EPS, MBSU, PEC power supply, PEC firing circuits, 
PEC capacitor banks, Flight Plug” 

- Faults and failures 

• “RIU Test Port Flight Cap failure” 

• “Short circuit, short” (or sneak circuit, fail open, race condition) 

• “Prevents power, redirects power” (or not deliver energy) 

• “No power reaches NSI” 

• Others: “fails to command” 
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XML Output for Model Reuse 


• XML file output of model information 
and traceability for use in other tools 

- Components, connections, and other 
model properties 

• XML output function uses an easily 
changed specification 

- Accommodates changes in the model 
structure or output properties 

• XML output for LAS pyrotechnics model 
delivered to Triakis 
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Virtual System Integration Lab (VSIL) 

AS Models 


a® 


El"Q:Tt'ie World; 

□ System 
| |- |P Orion CEV 

B O LAS 

q | Thermal Battery A 

■ Power Source 
Therm Batt NSI 
B H Thermal Battery B 
Pwr Bus Node 
Pwr Rtn Node 
± O Abort Motor 
EEL^J Jettison Motor 
E H Att Ctrl Motor 
- H LAS Pyros 

I Pyro Arm Plug A 
Pyro Arm Plug B 
AM Safe Arm 
E ■ NSI A 


LAS VSIL part tree 


| f 

i 


\+\ m NSI B 

Pwr Node 
Safe Load 
Arm Load 
JM Safe Arm 
Pwr Bus Node 
Pwr Rtn Node 
Pwr Out A Node 
Pwr Out A Rtn Node 
Pwr Out B Node 

Pwr Out B Rtn Node 

E □ LAS A VP 
[j- |P Crew Module 
CM Battery 
Single Pt Gnd 
Pwr Bus Node 
Pwr Rtn Node 
- |p CM Avionics 
+ ■ VMC 1 
E H CM RIU 1 
■ Pwr Bus Node 
| Pwr Rtn Node 


Crev 

I 


Example NSI part 

- Connections, 
interfaces and 

Input / Output 

- Inner models 

- Functions 
(Requirements) 

- Internal variables 

- Failures 


Nasa_Std_l niti ator 
NSI Load 
Resistor tri a 


28VDC_P\ 
Rre I 



DU Arg 280.000000 
PLUS_0 
DU Arg 0.025000 
MINUS_0 

PLJUSJ 

ITRIP 

MINUS I 


i!8VDC_PV\R_0 

l=ire_0 

.Trip 


Class Name Nana Std Initialor 

Parts: Class/Name 

Resistor tri a NSI Load 

l|rpiJl Signals: Name/Type 


28VDC PWR I 

SigThev 

Fire I 


Output Signals: Name/Type 


28VDC PWR O 

SigThev 

Fire O 

SigThev 

Trip 

SigBool 

Initiate 

SigBool 

Requirements 


1. The NSI shall initially be READY to fire. 

2. The NSI shall send out the Initiate signal 
upon receipt of a FIRE signal greater than 
7VDC @ 25mA is received. 

3 . The NSI shall change state to FIRED 
followingFIRE event. 

4 . Once FIRED, the NSI shall not change 
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Example LAS Pyro Nominal Test Results 


LAS Pyro Nominal Test Results 

Test l: LAS Pyro Safe/Flight Plug Tests 

Step la. Verify Safe/Flight Plug Initial State == SAFE 
Initial Plug A Status == SAFE: +++ PASS +++ 

Initial Plug B Status == SAFE: +++ PASS +++ 

Step lb. Verify Safe/Flight Plug output voltage <= 1.0 
Plug A output voltage == 0.187484: +++ PASS +++ 
Plug B output voltage == 0.187484: +++ PASS +++ 

Step lc. Change Safe/Flight Plug State to FLIGHT 
New Plug A State == FLIGHT: +++ PASS +++ 

New Plug B State == FLIGHT: +++ PASS +++ 

Step Id. Verify Safe/Flight Plug output voltage >=24.0 
Plug A output voltage == 27.996068: +++ PASS +++ 
Plug B output voltage == 27.996068: +++ PASS +++ 

Step le. Change Safe/Flight Plug State to SAFE 
New Plug A Status == SAFE: +++ PASS +++ 

New Plug B Status == SAFE: +++ PASS +++ 

Step If. Verify Safe/Flight Plug output voltage <= 1.0 
Plug A output voltage == 0.205305: +++ PASS +++ 
Plug B output voltage == 0.205305: +++ PASS +++ 


Test 2: LAS Safe/Arm Valve Tests 

Step 2a. Verify Safe/Arm Valve Initial State == SAFE 
Abort Motor Valve Status == SAFE: +++ PASS +++ 
Jettison Motor Valve Status == SAFE: +++ PASS +++ 

Step 2b. Setting: Command SA Valves to ARMED State 
Abort Motor Valve Status == ARMED: +++ PASS +++ 
Jettison Motor Valve Status == ARMED: +++ PASS +++ 

Test 2 Final Result: +++ PASSED +++ 


Test 1 Final Result: +++ PASSED +++ 
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HIT Models for Path Analysis 



• Hazard Identification Tool models are being 
enhanced for HIT-path analysis of LAS pyrotechnics 
paths and dependencies 

- Component mode transitions that are actions with enabling 
or disabling conditions (e.g., energy, power, percussion) 

• Most LAS operations of concern are mode transitions rather 
than continuous actions occurring within operating modes 

- Variable properties for entities transferred across connection 
paths (e.g., command signal values) 

• A simplified LAS pyrotechnics model was constructed 
to test these new capabilities 

- Templates for Pyrotechnic Devices, Pyro Event Controllers, 
Initiators, Power Supplies, Safe and Arm Devices 
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Model Reuse Study 
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Reusable Model Information 


• Component hierarchy and component- 
connection architecture 

- System modes 

- Configurations and phases 

• Functions and actions of components 

• Component modes/states and transitions 

- Operating and failure modes 

- State transitions and triggers 

• Faults and hazards 

- Disabled functions, actions and transitions 

• Instrumentation and key value constraints 
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Findings/Recommendations 



• Visualization can help SMA personnel 

- Overview and drill down to review large documents 

• Most useful documents are lIRDs, FMEAs, Hazard 
Reports at PDR 

- Pre-PDR documents are not mature enough 

- Extraction from requirements, structured text descriptions, 
structured worksheets and tables 

• PowerPoint charts and schematics are not promising 
formats for extraction 

• Standard requirements formats for model generation 
could help both authors and modelers 

• Model extractions can be reused for FSMs and 
TEAMS models 
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Planned Capability 



• Situations where these tools can be applied 

- Automatic extraction of Information for model development 
in aerospace programs and projects (Orion, Altair & others) 

• System components connections, interfaces, dependencies 

• Functions, actions, failures and hazards 

• Modes and states and transitions 

• Auxiliary information such as source text and traceability 
information 

- Development of low-fidelity early (PDR) models of systems 
interacting with software and controls 

- Development of visualizations for safety analysis 

- Analysis/simulation of system dependencies and paths of 
failure causes, effects and hazards 

• Identify scenarios for integration tests 
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Technical Solutions & Challenges 



• Technical Solutions 

- Improved extraction from structured text in more types of 
documents and document sections 

- More automation of construction of models and visualizations 
from extracted information 

- Specification files to handle changes in extraction, model 
construction, and XML model output 

• Improvements needed for wider use 

- Support for updating specification files 

- More support for manual interaction in model construction, 
review and expansion 

- Library of model templates for types of components 

• Methods for mapping model information to templates 

- Support for systematic path analysis 
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